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From: Riccardo Poli [R.Poli@cs.bham.ac.uk] 
Sent: Friday, January 15, 1999 6:44 AM 
To: koza@SMI.Stanford.EDU 
Cc: nordin@fy.chalmers.se 
Subject: eurogp99 review results 

Dear Author, 

Thank you very much for submitting a paper to the Second European 
Workshop on Genetic Programming (EuroGP'99). 

Based upon the reviewers' comments, we are very pleased to inform you 
that your paper has been accepted for ORAL presentation at the 
workshop and publication in the proceedings. 

in order to achieve the highest quality proceedings, we ask you to 
carefully consider the reviewers' comments appended to this message 
when preparing the final version of your contribution. Instructions on 
how to format the camera ready version of your paper are available 
from 

http : //www. spri nger . de/comp/1 ncs/authors . html 

where you will find instructions and style files for LaTex users and 
for Word users, instructions on how to send us the electronic version 
of your paper will be circulated next week. 

Please, note that the maximum length allowed for the camera ready 
version of your paper is 16 pages (everything included). 

Also, we would like to remind you that the deadline for sending the 
camera ready manuscript is 15 FEBRUARY 1999 and that this is a firm 
deadl i ne . 

Please forward this message to your co-authors (if any). 
Looking forward to seeing you in Goteborg. 
Best regards, 



Riccardo Poli and Peter Nordin 
EuroGP'99 Programme Co-chairs 
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EUROGP'99 PAPER REVIEW FORM 
Please return this form by email to: 
R.Poli@cs.bham.ac.uk AND nordin@fy.chalmers.se 
by Monday 11th January 1999 



REVIEWER NUMBER: 10 

TITLE: Genetic Programming as a Darwininan Invention Machine 
AUTHOR(S): Koza, Bennett and Stiff el man 



REVIEWER'S STATEMENT: 

- I have read this paper. 
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- I am not at the same institution as any of the authors. 

- I am not reviewing more than one paper for this conference by this author. 

- I do not have an especially close relationship (or an antagonistic 
relationship) with the author(s). 

- I am not re-reviewing this work (i.e., I have not, within the past year, 
reviewed substantially the same work by the same author). 

- This review is my own opinion and work and has not been delegated to 
anyone (specifically including students or subordinates). 

- I have not discussed my review of this paper with any other peer reviewer 
for this conference. 

- I will regard any new intellectual content of this paper as confidential 
and will not disclose or use it until such time as it gets published under 
the author's name or otherwise becomes publicly known (and, if I discussed 
this review with anyone else at all, they have also explicitly agreed to 
maintain the author's work as confidential and I notified the program chairs 
of their agreement to do so). 

- I feel that I am qualified to review this paper. 

XX Yes (indicates a "yes" to all of the above items for this particular paper) 
No 



9 8 7 6 5 4 3 2 1 0 
ORIGINALITY : 3 

SIGNIFICANCE OF PROBLEM : 3 
TECHNICAL SOUNDNESS : 9 

CLARITY AND ORGANISATION : 8 
RELEVANCE TO CONFERENCE : 6 

YOUR recommendation: (delete all but one of these) 
o definitely accept as poster 

REVIEWER'S CONFIDENCE: 

o high 

AMOUNT OF REVISION NEEDED: 

o none 

FORMAT: Does the paper conform fully with the EuroGP'99 proceedings 
length (12 pages, 15 max) and format (Springer Verlag's 
Lecture Notes in Computer Science, see the example at 
http : //www. cs . bham . ac . uk/~rmp/eebi c/1 1 ncs_exampl e . ps) ? 

Please delete as necessary. 

o The paper is NOT fully compliant for the following reasons: 

- The printing area is NOT 12.2cm x 19.3cm (4.8 x 7.6 inches) 

- The font is NOT Computer Modern lOpt (looks smaller) 

Other problems (please specify): 

This paper probably would be too long if formatted with the required page size. 

ENGLISH: Please delete as necessary. 

o The English is fine. 
Additional comments: 
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COMMENTS FOR AUTHOR(S) : 



The paper describes the use of GP with an ontogenic representation to design an 
analog lowpass filter circuit, it describes how to use subgraph isomorphism 
penalties to lead evolution toward novel solutions. 

It is ironic that so much of this paper seems to be derived heavily from earlier 
papers. I hope the author didn't include large sections of their earlier papers 
verbatim, but didn't check. It is ironic that a paper on innovation would have so 
much familiar verbiage in it. 

Some of this isn't needed. For example, this is at least the third paper from these 
authors where I've seen the two paragraph quotation from Aaserud and Nielson to the 
effect that "designing analog circuits is hard". Since that isn't even the point 
here, perhaps it would suffice just to SAY "designing analog circuits is hard" and 
cite Asserud and Nielsen. Much of chapter 2 seems unnecessary, in fact, and a 
simple reference to earlier work should suffice. 

One part of the paper was literally unreadable: the text above Figure 1 was masked 
out by the figure, somehow. 

The discussion of patent law was very helpful and interesting. 

The main contribution of the paper was a technique for penalizing evolved solutions 
if they were too similar to known solutions. Specifically, to drive the solutions 
away from a ladder architecture, individuals were penalized for being isomorphic to 
subgraphs of a large ladder of capacitors. 

But this is in actuality little more than old-fashioned fitness sharing, where the 
designs to be avoided are seeded into the population. The main claim of the paper 
then reduces to this: fitness sharing creates diversity. That is hardly a new 
result. This is why I ranked the significance and originality rather low. 

This technique also is unlikely to generalize. For example, a T structured circuit 
cannot be artificially enlarged, as the authors enlarged their ladder. 
Consequently, searching for subgraphs that are isomorphic to an evolved circuit 
cannot be done for T structured circuits, in fact, it is difficult to see how this 
subgraph isomorphism checking approach can work for any type of prior art other than 
those with repetitive simple units in a simple structure, where further repetition 
only fine tunes the circuit. Given this criticism, the authors are claiming more 
for this approach than is warranted. 

The title is also too grandiose. This is more a paper about using GP to find 
solutions with a small, highly restricted set of solutions to avoid, than a paper 
demonstrating a "Darwinian Invention Machine". Perhaps a better title would be "GP 
to find Patentable Circuits"? 

Nonetheless, the paper is well written. The use of GP to design analog circuits 
really is significant, because the problem really is hard (though lowpass filters 
are pretty well solved already). The idea of directing evolutionary innovation, 
even if the only real direction is "not here", is a good one. This work should be 
presented. 
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Please return this form by email to: 
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R.Poli@cs.bham.ac.uk AND nordin@fy.chalmers.se 

by Monday 11th January 1999 

REVIEWER NUMBER: 2 

TITLE: Genetic Programming as a Darwinian Invention Machine 
AUTHOR(S) : Koza 

REVIEWER'S STATEMENT: 

- I have read this paper. 

- I am not at the same institution as any of the authors. 

- I am not reviewing more than one paper for this conference by this author. 

- I do not have an especially close relationship (or an antagonistic 
relationship) with the author(s). 

- I am not re-reviewing this work (i.e., I have not, within the past year, 
reviewed substantially the same work by the same author). 

- This review is my own opinion and work and has not been delegated to 
anyone (specifically including students or subordinates). 

- I have not discussed my review of this paper with any other peer reviewer 
for this conference. 

- I will regard any new intellectual content of this paper as confidential 
and will not disclose or use it until such time as it gets published under 
the author's name or otherwise becomes publicly known (and, if I discussed 
this review with anyone else at all, they have also explicitly agreed to 
maintain the author's work as confidential and I notified the program chair 

s 

of their agreement to do so). 

- I feel that I am qualified to review this paper. 

Y_ Yes (indicates a "yes" to all of the above items for this particular paper 

Page 4 



Euro-GP-99-review-resul ts . txt 



) 



GOOD 



OK 



BAD 



9 8 7 6 5 4 3 2 1 0 



ORIGINALITY 



9 



SIGNIFICANCE OF PROBLEM 



9 



TECHNICAL SOUNDNESS 



9 



CLARITY AND ORGANISATION 



8 



RELEVANCE TO CONFERENCE 



9 



YOUR recommendation: (delete all but one of these) 

o definitely accept as paper (talk) 

REVIEWER'S CONFIDENCE: 

o high 

AMOUNT OF REVISION NEEDED: 

o moderate 

FORMAT: Does the paper conform fully with the EuroGP'99 proceedings 
length (12 pages, 15 max) and format (Springer Verlag's 
Lecture Notes in Computer Science, see the example at 
http : //www. cs . bham . ac . uk/~rmp/eebi c/1 1 ncs_exampl e . ps) ? 

Please delete as necessary. 

o The paper is NOT fully compliant for the following reasons: 

- The printing area is NOT 12.2cm x 19.3cm (4.8 x 7.6 inch 
Other problems (please specify): 
Not using LNCS standard at all. Eg rules for referring to Fig., Sect., 
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ENGLISH : Please delete as necessary. 



o The English is fine. 
Additional comments: 



COMMENTS FOR AUTHOR(S) : 

A important aspect of this paper is the use of a similarity measure to 
discourage GP from finding solutions similar to existing 
ones. Perhaps this should be made more explicit in the title? 

A possibility for future work would be to include dissimilarity 
measures within the fitness measure rewarding members of the 
population that are different from others in the same population. 
Thus the dissimilarity could itself be evolved. 

Co-evolutionary and mul ti -objective (possibly with fitness sharing) 
approaches come to mind. 

p3 sec 3 para 3 

formatting problems with text immediately above figure 1. 

p5 sec 4.5 para 3 

Equation F(t) has ugly fonts 

p6 top text 

How does algorithm scale with graph size? 

p6 end 1st para 

Ref to Koza 1995; looks wrong 

p6 4.8 bot 

Say more explicitly how the 600 where chosen. 
Eg are they the best 600? 

Say (presumably) 4 copies of each sent. (Do you cite why you use 2%) 
Presumably each node runs asynchronous, any way include in paper. 

p7 fig4 

May be give each a separate number, ie Figs 4 5 and 6 
Some thing messed up with caption on 4b. 



p7 sec 5 2nd para 2nd sentence 

I think "very little" is over stating it. 

Fig 4a shows firstly that it _is_ a low bandpass filter 

and secondly its cut off (50%) is close(ish) to the desired point. 



p9 2nd para 

is spelling of Geottingen right? 

p9 sec 6 

mention runtime (may be also comparison of cost of genius v 
cost of hardware?) 
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EUROGP'99 PAPER REVIEW FORM 
Please return this form by email to: 
R.Poli@cs.bham.ac.uk AND nordin@fy.chalmers.se 
by Monday 11th January 1999 



REVIEWER NUMBER: 8 

TITLE: Genetic Programming as a Darwinian Invention Machine 
AUTHOR(S): John R. Koza, Forrest H Bennet III, Oscar Stiffelman 

REVIEWER'S STATEMENT: 

- I have read this paper. 

- I am not at the same institution as any of the authors. 

- I am not reviewing more than one paper for this conference by this 
author. 

- I do not have an especially close relationship (or an antagonistic 
relationship) with the author(s). 

- I am not re-reviewing this work (i.e., I have not, within the past 
year, 

reviewed substantially the same work by the same author). 

- This review is my own opinion and work and has not been delegated to 
anyone (specifically including students or subordinates). 

- I have not discussed my review of this paper with any other peer 
reviewer 

for this conference. 

- I will regard any new intellectual content of this paper as 
confidential 

and will not disclose or use it until such time as it gets published 
under 

the author's name or otherwise becomes publicly known (and, if I 
di scussed 

this review with anyone else at all, they have also explicitly agreed 

to 

maintain the author's work as confidential and I notified the program 
chai rs 

of their agreement to do so). 

- I feel that I am qualified to review this paper. 

X_ Yes (indicates a "yes" to all of the above items for this particular 
paper) 



GOOD OK BAD 

9 8 7 6 5 4 3 2 1 0 
ORIGINALITY : 8 

SIGNIFICANCE OF PROBLEM : 6 
TECHNICAL SOUNDNESS : 6 

CLARITY AND ORGANISATION : 5 
RELEVANCE TO CONFERENCE : 8 

YOUR recommendation: (delete all but one of these) 

o definitely accept as paper (talk) 

REVIEWER'S CONFIDENCE: 

o medium 



AMOUNT OF REVISION NEEDED: 

o little 
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FORMAT: Does the paper conform fully with the EuroGP'99 proceedings 
length (12 pages, 15 max) and format (Springer Verlag's 
Lecture Notes in Computer Science, see the example at 
http : //www. cs . bham . ac . uk/~rmp/eebi c/1 1 ncs_exampl e . ps) ? 

Please delete as necessary. 

o The paper is NOT fully compliant for the following reasons: 

- The printing area is NOT 12.2cm x 19.3cm (4.8 x 7.6 

i nches) 

- The paper is NOT (fully) single column 

- Title, authors and abstract ARE in a separate page 

Other problems (please specify): 

- Above figures 1 and 2 a part of the preceeding text 
is deleted. There seem to be some trouble with the 
figure placement. 

- The references should be numbered. 

ENGLISH: Please delete as necessary, 
o The English is fine. 
Additional comments: 
COMMENTS FOR AUTHOR(S) : 



Main points: 



The idea of automatically creating analog cirtcuits by GP is 
very fascinating. However I do not see a great motivation 
in avoiding existing key characteristics. Much more important 
to me seems to be for example 

- the size of the circuit (number of C and L) 

- the power consumption 

in fact, in the end of section 5 you say "This reduction [of one 
component] was very important in the days when filters in 
telephones were manufactured by individually soldering in expensive 
discrete components". So why don't you use GP to evolve a circuit 
that meets the required frequency behavior with a minimum nuber 
of components? 

- You don't say in which generation the eight circuits of table 1 
occured. Or did you run the GP for all 501 generations? (This 
would mean that you tried about one billion circuits!) 

- in table 1 row 5 there seems to be a typo as the overall fitness 
is the same as the frequency response factor. 

Minor points: 



- The equation in section 4.5 looks quite unreadable. Why 
do you use such an enlarged font? 

- What is the meaning of the variable $ in the same formula? 

- Figure 3 is not numbered and has no caption. 

- The better citation for Gruau 1992 (technical report) is his 
Phd: "Neural Network Synthesis using Cellular Encoding and 
the Genetic Algorithm". University of Grenoble, France, 1994 
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